Jmni/zarr validation - #607
Conversation
Calibration is a measurement of the rig that recorded one episode, so it has to travel with the episode instead of living in a class constant. Add the `calibration` attribute block and the single reader every consumer goes through. The block names one reference frame and expresses every pose in it: `cameras[c].ref_T_cam` is a camera pose, `arm_bases[side]` is `ref_T_armbase`. `Calibration.base_T_cam(side)` composes the two into what the EVA transform pipeline consumes. No stored episode needs a rewrite. `read_calibration` prefers the block and falls back to lifting the legacy `intrinsics`/`extrinsics` pair, whose per-arm values are the front camera's pose in each arm base, so the lifted reference frame is `camera:front_1` and `arm_bases` is their inverse. `ZarrWriter` writes the block and derives the legacy pair from it, so readers that predate the block keep working. `ZarrEpisode`, `ZarrDataset` and the inspector now read through the shim rather than parsing the attributes themselves, which also drops two duplicate 3x3-to-3x4 normalizations. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
`intrinsics` only had to be non-empty, so `eva_to_zarr.py` calibrates one camera while writing three image streams and passes. Coverage closes that: the declared `images.<camera>` keys are checked against the cameras that carry a K. The rule is opt-in. Every EVA episode in the corpus fails it today, so `strict=False` warns and `strict=True` raises. Flipping the default before the corpus is fixed would turn a good check into an outage. `uncalibrated_cameras` lives beside the calibration reader so the validator can reuse the same rule rather than restate it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
The transform pipeline expressed actions in the camera frame using `Eva.EXTRINSICS`, a class constant, and ignored what the episode stored. Two vendors on one platform have two rigs, and a class constant cannot tell them apart, so the factorization is unsound until the training path reads the episode. `ZarrDataset` now puts each arm's `base_T_cam` from the episode calibration into every sample, and a transform's `extra_batch_key` becomes a fallback that fills only keys the sample does not already carry. `Eva.EXTRINSICS` still covers an episode that declares no extrinsics. This changes numbers only for an episode whose stored rig differs from the constant. A test asserts the two agree: an episode holding `Eva.EXTRINSICS` and an episode holding none produce the same actions, which is every EVA episode in the corpus today. Two further tests show a second rig moves the actions and lands exactly where that rig predicts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
Collecting calibration is cheap and time-critical; consuming it is expensive and deferrable. A vendor who ships 10k episodes and later turns out to need distortion coefficients cannot supply them, because the rig has moved. Each camera in the block now declares `model` and `distortion`, defaulting to PINHOLE with no coefficients. The coefficient count is checked against the model, so a KANNALA_BRANDT camera with five coefficients or a pinhole camera with any is an error at ingest. No projection site reads either field. Nothing under `algo/` or `models/` reads intrinsics at all, so this changes no trained weight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
`python -m egomimic.rldb.zarr.validate <path> [--strict]` checks one episode
against `schema/episode_v3.yaml`. The rules live in the schema file; the module
reads them and holds none of its own, so a contributor can read and diff the
contract instead of reverse-engineering it from Python.
It replaces the 300-line `validate_episode()` pasted into CONTRIBUTING_DATA.md,
which is the only validator a contributor can run today and which passed an
all-identity episode with 28 checks OK and 0 errors.
Array rules resolve their widths through the registry, so `arm_dof` and the
keypoint topology come from `platforms.yaml` and `end_effectors.yaml` rather
than being restated. `{side}` expands over the arms the arity declares, and a
rule applies only when its `when` conditions hold, so a human episode is never
asked for a gripper.
Rules the corpus does not meet yet are declared `required: strict`: a warning
by default and an error under `--strict`. Flipping them first would turn a good
check into an outage, since the intrinsics coverage rule alone fails every
`eva_fold` episode.
`total_frames` stays the sole authoritative length: axis 0 is a lower bound so
a padded tail passes, and every other axis is exact.
Adds `ResolvedEmbodiment.arity` and `.sides`, which the array rules need to
know whether an episode owes one arm or two.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
Six rules that turn the failure modes we have actually been shipped into findings at ingest instead of a policy that trains badly six weeks later. - pose_degeneracy: a track that never moves, or that holds an exact identity rotation on more than 1% of frames, is a placeholder, not a measurement. - calibration_degeneracy: an exactly-identity extrinsic puts the camera at the arm base. It survives every other check, because the overlay it draws still looks plausible, bunched at the image centre. - intrinsics_signature: `fx == fy == W` with the principal point at the exact image centre is a synthesized camera. The `fx == W` conjunct is what keeps our own rectified Aria K, fx 266.5 at W 640, out of the net. - timestamps: strictly increasing, and a warning when the step is a power of two of at least 64 ns, which is float64 seconds converted to nanoseconds. A second stored time base is an error. - annotation_coverage: annotations cover 90% of the episode or the tail is trimmed. Opt-in, since EVA episodes pass no annotations at all. - annotation_text: no delimiter-encoded metadata. A skill taxonomy defined from one vendor's data is a taxonomy we will regret. Every rule and its threshold lives in the schema file. Run against the Sharpa sample the rules reproduce each measurement in the review by hand: 370 identity frames on all three pose tracks, identity extrinsics, the synthesized 480-pixel K on all three cameras, three duplicate timestamps under 256 ns quantization, 76% annotation coverage, and the ` | Skill: pick` suffix. The validator that shipped in CONTRIBUTING_DATA.md passes that episode with 28 checks OK and 0 errors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
A sample sent to show the shape of a delivery is real data in the schema sense and not real data in the training sense, and until now nothing downstream could tell the two apart. Sharpa reached for this with six free-text `*_status` attributes that nothing reads. `data_status` is `complete` or `structural_sample`. The writer records it and rejects anything else. A non-complete episode gets no staging row and is skipped by the resolver, so it cannot reach a training run by any path. An episode written before the attribute existed reads as `complete`: the corpus predates the distinction and every episode in it was delivered as finished data. The validator therefore asks for the attribute only under `--strict`, while an unknown value is always an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012rCh1QmnyqgPZ5TJpsfMQb
Claude Code ReviewReview of PR #607: Jmni/zarr validationSummaryThis PR introduces an embodiment registry system (YAML-driven platforms/end-effectors/aliases), a per-episode calibration block with a legacy shim, coordinate-convention docs ( Key concerns1. Scope mismatch with PR descriptionThe description says "per-episode calibration with fallback" and "structural validation," but the diff also renames extrinsics semantics repo-wide ( 2. Silent semantic change in
|
A validation script with the heuristic checks:
The following are checks configurable to be optional: